Iterative Methods for Linear Systems

This chapter contains an overview of some of the important techniques used to solve linear systems of equations Ax = b (1) by iterative methods. We consider methods based on two general ideas, splittings of the coeecient matrix, leading to stationary iterative methods, and Krylov subspace methods. These two ideas can also be combined to produce preconditioned iterative methods. In addition, we outline some convergence results for using the methods considered to solve two classes of model problems arising from elliptic partial diierential equations. In x1, we introduce the basic ideas of stationary iterative methods and consider several particular examples of such methods: the Jacobi, Gauss-Seidel, SOR and SSOR methods. We outline some results on convergence of these methods, for both general matrices and those with special structure. In x2, we give an overview of Krylov subspace methods for systems where the coeecient matrix is symmetric. These include the conjugate gradient method for symmetric positive-deenite systems, and several generalizatons of this technique for the symmetric indee-nite case. In x3, we examine the use of Krylov subspace methods for nonsymmetric problems. This is an active area of current research, and we highlight GMRES, the most popular method in current use, together with the QMR method, one of several new ideas being studied. In x4, we present several preconditioning techniques that can be used in combination with Krylov subspace methods. Our emphasis here is methods such as incomplete factorizations that are deened purely in terms of the algebraic structure of the coeecient matrix. In x5, we outline the convergence properties of the methods presented for two classes of model problems, the discrete Poisson equation , which is symmetric positive-deenite, and the discrete convection-diiusion equation, which is nonsymmetric. Finally, in x6, we present a brief discussion of several important topics that we have not considered here. Before proceeding, we introduce several points of notation. We will assume that A is a nonsingular real matrix of order n. All the methods considered generate a sequence of iterates x (k) that are intended to converge to x = A ?1 b. They all require a stopping criterion that can be used to determine when the iterate is suuciently accurate. We will not address this question in any detail, except to note that the residual r (k) = b ? Ax (k) is easily computable; a commonly used stopping criterion is to require that the relative …